Institutional boundaries and the challenges of aligning science advice and policy dynamics: the UK and Canada in the time of COVID-19.

This comparison of institutions of science advice during COVID-19 between the Westminster systems of England/UK and Ontario/Canada focuses on the role of science in informing public policy in two central components of the response to the pandemic: the adoption of non-pharmaceutical interventions (NPIs) and the procuring of vaccines. It compares and contrasts established and purpose-built bodies with varying degrees of independence from the political executive, and shows how each attempted to manage the tensions between scientific and governmental logics of accountability as they negotiated the boundary between science and policy. It uses the comparison to suggest potential lessons about the relative merits and drawbacks of different institutional arrangements for science advice to governments in an emergency.

Ontario's COVID-19 Modelling Consensus Table: mobilizing scientific expertise to support pandemic response.

SETTING: COVID-19 has highlighted the need for credible epidemiological models to inform pandemic policy. Traditional mechanisms of commissioning research are ill-suited to guide policy during a rapidly evolving pandemic. At the same time, contracting with a single centre of expertise has been criticized for failing to reflect challenges inherent in specific modelling approaches. INTERVENTION: This report describes an alternative approach to mobilizing scientific expertise. Ontario's COVID-19 Modelling Consensus Table (MCT) was created in March 2020 to enable rapid communication of credible estimates of the impact of COVID-19 and to accelerate learning on how the disease is spreading and what could slow its transmission. The MCT is a partnership between the province and academic modellers and consists of multiple groups of experts, health system leaders, and senior decision-makers. Armed with Ministry of Health data, the MCT meets once per week to share results from modelling exercises, generate consensus judgements of the likely future impact of COVID-19, and discuss decision-makers' priorities. OUTCOMES: The MCT has enabled swift access to data for participants, a structure for developing consensus estimates and communicating these to decision-makers, credible models to inform health system planning, and increased transparency in public reporting of COVID-19 data. It has also facilitated the rapid publication of research findings and its incorporation into government policy. IMPLICATIONS: The MCT approach is one way to quickly draw on scientific advice outside of government and public health agencies. Beyond speed, this approach allows for nimbleness as experts from different organizations can be added as needed. It also shows how universities and research institutes have a role to play in crisis situations, and how this expertise can be marshalled to inform policy while respecting academic freedom and confidentiality.

RéSUMé: LIEU: La COVID-19 a mis en évidence le besoin de modèles épidémiologiques crédibles pour éclairer la politique pandémique. Les mécanismes habituels pour commander des travaux de recherche sont peu propices à orienter les politiques lors d'une pandémie qui évolue rapidement. En même temps, la passation de contrats avec un seul centre d'expertise est critiquée, car elle ne tient pas compte des difficultés inhérentes de certaines approches de modélisation. INTERVENTION: Le présent rapport décrit une approche de rechange pour mobiliser le savoir scientifique. L'Ontario a créé en mars 2020 une Table de concertation sur la modélisation (TCM) qui permet de communiquer de façon rapide et fiable les estimations des effets de la COVID-19 et d'apprendre plus vite comment la maladie se propage et ce qui pourrait en ralentir la transmission. La TCM, un partenariat entre les modélisateurs de la province et des milieux universitaires, est composée de plusieurs groupes d'experts, de dirigeants du système de santé et de décideurs de haut niveau. Armée des données du ministère de la Santé, la TCM se réunit une fois par semaine pour partager les résultats d'exercices de modélisation, générer des jugements consensuels sur les futurs effets probables de la COVID-19 et discuter des priorités des décideurs. RéSULTATS: La TCM rend possible un accès rapide aux données pour les participants, une structure pour élaborer des estimations consensuelles et les communiquer aux décideurs, des modèles fiables pour éclairer la planification du système de santé, ainsi qu'une transparence accrue dans la communication des données sur la COVID-19 au public. Elle facilite aussi la publication rapide des résultats de recherche et leur intégration dans la politique gouvernementale. CONSéQUENCES: L'approche de la TCM est un moyen d'obtenir rapidement des conseils scientifiques à l'extérieur du gouvernement et des organismes de santé publique. Au-delà de sa rapidité, cette approche offre une grande souplesse, car des experts de différents organismes peuvent être ajoutés au besoin. Elle montre aussi que les universités et les établissements de recherche ont un rôle à jouer dans les situations de crise, et qu'il est possible de mobiliser leurs compétences pour éclairer les politiques tout en respectant la liberté et la confidentialité des milieux de la recherche et de l'enseignement.

Are we prepared? The development of performance indicators for public health emergency preparedness using a modified Delphi approach.

BACKGROUND: Disasters and emergencies from infectious diseases, extreme weather and anthropogenic events are increasingly common. While risks vary for different communities, disaster and emergency preparedness is recognized as essential for all nation-states. Evidence to inform measurement of preparedness is lacking. The objective of this study was to identify and define a set of public health emergency preparedness (PHEP) indicators to advance performance measurement for local/regional public health agencies. METHODS: A three-round modified Delphi technique was employed to develop indicators for PHEP. The study was conducted in Canada with a national panel of 33 experts and completed in 2018. A list of indicators was derived from the literature. Indicators were rated by importance and actionability until achieving consensus. RESULTS: The scoping review resulted in 62 indicators being included for rating by the panel. Panel feedback provided refinements to indicators and suggestions for new indicators. In total, 76 indicators were proposed for rating across all three rounds; of these, 67 were considered to be important and actionable PHEP indicators. CONCLUSIONS: This study developed an indicator set of 67 PHEP indicators, aligned with a PHEP framework for resilience. The 67 indicators represent important and actionable dimensions of PHEP practice in Canada that can be used by local/regional public health agencies and validated in other jurisdictions to assess readiness and measure improvement in their critical role of protecting community health.

Out of sync: a Shared Mental Models perspective on policy implementation in healthcare.

The impact of policy ambiguity on implementation is a perennial concern in policy circles. The degree of ambiguity of policy goals and the means to achieve them influences the likelihood that a policy will be uniformly understood and implemented across implementation sites. We argue that the application of institutional and organisational theories to policy implementation must be supplemented by a socio-cognitive lens in which stakeholders' interpretations of policy are investigated and compared. We borrow the concept of 'Shared Mental Models' from the literature on industrial psychology to examine the microprocesses of policy implementation. Drawing from interviews with 45 key informants involved in the implementation of a hospital funding reform, known as Quality-Based Procedures in Ontario, Canada, we identify divergent mental models and explain how these divergences may have affected implementation and change management. We close with considerations for future research and practice.

A population-based approach to integrated healthcare delivery: a scoping review of clinical care and public health collaboration.

BACKGROUND: A population-based approach to healthcare goes beyond the traditional biomedical model and addresses the importance of cross-sectoral collaboration in promoting health of communities. By establishing partnerships across primary care (PC) and public health (PH) sectors in particular, healthcare organizations can address local health needs of populations and improve health outcomes. The purpose of this study was to map a series of interventions from the empirical literature that facilitate PC-PH collaboration and develop a resource for healthcare organizations to self-evaluate their clinical practices and identify opportunities for collaboration with PH. METHODS: A scoping review was designed and studies from relevant peer-reviewed literature and reports between 1990 and 2017 were included if they met the following criteria: empirical study methodology (quantitative, qualitative, or mixed methods), based in US, Canada, Western Europe, Australia or New Zealand, describing an intervention involving PC-PH collaboration, and reporting on structures, processes, outcomes or markers of a PC-PH collaboration intervention. RESULTS: Out of 2962 reviewed articles, 45 studies with interventions leading to collaboration were classified into the following four synergy groups developed by Lasker's Committee on Medicine and Public Health: Coordinating healthcare services (n = 13); Applying a population perspective to clinical practice (n = 21); Identifying and addressing community health problems (n = 19), and Strengthening health promotion and health protection (n = 21). Furthermore, select empirical examples of interventions and their key features were highlighted to illustrate various approaches to implementing collaboration interventions in the field. CONCLUSIONS: The findings of our review can be utilized by a range of organizations in healthcare settings across the included countries. Furthermore, we developed a self-evaluation tool that can serve as a resource for clinical practices to identify opportunities for cross-sectoral collaboration and develop a range of interventions to address unmet health needs in communities; however, the generalizability of the findings depends on the evaluations conducted in individual studies in our review. From a health equity perspective, our findings also highlight interventions from the empirical literature that address inequities in care by targeting underserved, high-risk populations groups. Further research is needed to develop outcome measures for successful collaboration and determine which interventions are sustainable in the long term.

The relationship between hospital adoption and use of high technology medical imaging and in-patient mortality and length of stay.

PURPOSE: The purpose of this paper is to investigate the relationship between hospital adoption and use of computed tomography (CT) scanners, and magnetic resonance imaging (MRI) machines and in-patient mortality and length of stay. DESIGN/METHODOLOGY/APPROACH: This study used panel data (2007-2010) from 124 hospital corporations operating in Ontario, Canada. Imaging use focused on medical patients accounting for 25 percent of hospital discharges. Main outcomes were in-hospital mortality rates and average length of stay. A model for each outcome-technology combination was built, and controlled for hospital structural characteristics, market factors and patient characteristics. FINDINGS: In 2010, 36 and 59 percent of hospitals had adopted MRI machines and CT scanners, respectively. Approximately 23.5 percent of patients received CT scans and 3.5 percent received MRI scans during the study period. Adoption of these technologies was associated with reductions of up to 1.1 percent in mortality rates and up to 4.5 percent in length of stay. The imaging use-mortality relationship was non-linear and varied by technology penetration within hospitals. For CT, imaging use reduced mortality until use reached 19 percent in hospitals with one scanner and 28 percent in hospitals with 2+ scanners. For MRI, imaging use was largely associated with decreased mortality. The use of CT scanners also increased length of stay linearly regardless of technology penetration (4.6 percent for every 10 percent increase in use). Adoption and use of MRI was not associated with length of stay. RESEARCH LIMITATIONS/IMPLICATIONS: These results suggest that there may be some unnecessary use of imaging, particularly in small hospitals where imaging is contracted out. In larger hospitals, the results highlight the need to further investigate the use of imaging beyond certain thresholds. Independent of the rate of imaging use, the results also indicate that the presence of CT and MRI devices within a hospital benefits quality and efficiency. ORIGINALITY/VALUE: To the authors' knowledge, this study is the first to investigate the combined effect of adoption and use of medical imaging on outcomes specific to CT scanners and MRI machines in the context of hospital in-patient care.

Lessons From the Canadian Experience With Single-Payer Health Insurance: Just Comfortable Enough With the Status Quo.

With single-payer public health insurance again on the political radar in the United States at both the state (California) and federal (Democrat party) levels, the performance of the Canadian health care system during the last 50 years and the lessons it may offer should be considered. Canadians are proud of their universal approach to health insurance based on need rather than income. The system has many strengths, such as the ease of obtaining care, relatively low costs, and low administrative costs, with effectiveness and safety roughly on par with other countries, including those, such as the United States, that spend considerably more per capita. There are increasing frustrations, however, with system performance, especially with issues related to access and coordination of care. Medicine has changed dramatically since the introduction of Canadian Medicare in the late 1960s, which primarily covered acute care physician and hospital services-the needs of the time. Meaningful reforms that match coverage and services to changing needs, especially those of community-based patients with multiple chronic conditions, have been difficult to implement. The status quo represents a compromise struck decades ago between payers and physicians and organizations that provide health care, and the current system works just well enough for those who both need it and vote. Enacting substantial change simply carries too much risk. Perhaps the most important lesson that the United States can learn from Canada's experience during the last 50 years is that a single-payer health care system solves a lot of problems, but it does not equate to an integrated, well-managed system that can readily meet the changing health care needs of a population.

Standardising costs or standardising care? Qualitative evaluation of the implementation and impact of a hospital funding reform in Ontario, Canada.

BACKGROUND: Since 2011, the Government of Ontario, Canada, has phased in hospital funding reforms hoping to encourage standardised, evidence-based clinical care processes to both improve patient outcomes and reduce system costs. One aspect of the reform - quality-based procedures (QBPs) - replaced some of each hospital's global budget with a pre-set price per episode of care for patients with specific diagnoses or procedures. The QBP initiative included publication and dissemination of a handbook for each of these diagnoses or procedures, developed by an expert technical group. Each handbook was intended to guide hospitals in reducing inappropriate variation in patient care and cost by specifying an evidence-based episode of care pathway. We explored whether, how and why hospitals implemented these episode of care pathways in response to this initiative. METHODS: We interviewed key informants at three levels in the healthcare system, namely individuals who conceived and designed the QBP policy, individuals and organisations supporting QBP adoption, and leaders in five case-study hospitals responsible for QBP implementation. Analysis involved an inductive approach, incorporating framework analysis to generate descriptive and explanatory themes from data. RESULTS: The 46 key informants described variable implementation of best practice episode of care pathways across QBPs and across hospitals. Handbooks outlining evidence-based clinical pathways did not address specific barriers to change for different QBPs nor differences in hospitals' capacity to manage change. Hospitals sometimes found it easier to focus on containing and standardising costs of care than on implementing standardised care processes that adhered to best clinical practices. CONCLUSION: Implementation of QBPs in Ontario's hospitals depended on the interplay between three factors, namely complexity of changes required, internal capacity for organisational change, and availability and appropriateness of targeted external facilitators and supports to manage change. Variation in these factors across QBPs and hospitals suggests the need for more tailored and flexible implementation supports designed to fit all elements of the policy, rather than one-size-fits-all handbooks alone. Without such supports, hospitals may enact quick fixes aimed mainly at preserving budgets, rather than pursue evidence- and value-based changes in care management. Overestimating hospitals' change management capacity increases the risk of implementation failure.

Adoption of high technology medical imaging and hospital quality and efficiency: Towards a conceptual framework.

Measuring the value of medical imaging is challenging, in part, due to the lack of conceptual frameworks underlying potential mechanisms where value may be assessed. To address this gap, this article proposes a framework that builds on the large body of literature on quality of hospital care and the classic structure-process-outcome paradigm. The framework was also informed by the literature on adoption of technological innovations and introduces 2 distinct though related aspects of imaging technology not previously addressed specifically in the literature on quality of hospital care: adoption (a structural hospital characteristic) and use (an attribute of the process of care). The framework hypothesizes a 2-part causality where adoption is proposed to be a central, linking factor between hospital structural characteristics, market factors, and hospital outcomes (ie, quality and efficiency). The first part indicates that hospital structural characteristics and market factors influence or facilitate the adoption of high technology medical imaging within an institution. The presence of this technology, in turn, is hypothesized to improve the ability of the hospital to deliver high quality and efficient care. The second part describes this ability throughout 3 main mechanisms pointing to the importance of imaging use on patients, to the presence of staff and qualified care providers, and to some elements of organizational capacity capturing an enhanced clinical environment. The framework has the potential to assist empirical investigations of the value of adoption and use of medical imaging, and to advance understanding of the mechanisms that produce quality and efficiency in hospitals.

Correction: Qualitative analysis of the dynamics of policy design and implementation in hospital funding reform.

[This corrects the article DOI: 10.1371/journal.pone.0191996.].

Development of Enriched Core Competencies for Health Services and Policy Research.

OBJECTIVE: To develop an enriched set of core competencies for health services and policy research (HSPR) doctoral training that will help graduates maximize their impact across a range of academic and nonacademic work environments and roles. DATA SOURCES/STUDY SETTING: Data were obtained from multiple sources, including literature reviews, key informant interviews, stakeholder consultations, and Expert Working Group (EWG) meetings between January 2015 and March 2016. The study setting is Canada. STUDY DESIGN: The study used qualitative methods and an iterative development process with significant stakeholder engagement throughout. DATA COLLECTION/EXTRACTION METHODS: The literature reviews, key informant interviews, existing data on graduate career trajectories, and EWG deliberations informed the identification of career profiles for HSPR graduates and the competencies required to succeed in these roles. Stakeholder consultations were held to vet, refine, and validate the competencies. PRINCIPAL FINDINGS: The EWG reached consensus on six sectors and eight primary roles in which HSPR doctoral graduates can bring value to employers and the health system. Additionally, 10 core competencies were identified that should be included or further emphasized in the training of HSPR doctoral students to increase their preparedness and potential for impact in a variety of roles within and outside of traditional academic workplaces. CONCLUSION: The results offer an expanded view of potential career paths for HSPR doctoral graduates and provide recommendations for an expanded set of core competencies that will better equip graduates to maximize their impact on the health system.

Qualitative analysis of the dynamics of policy design and implementation in hospital funding reform.

BACKGROUND: As in many health care systems, some Canadian jurisdictions have begun shifting away from global hospital budgets. Payment for episodes of care has begun to be implemented. Starting in 2012, the Province of Ontario implemented hospital funding reforms comprising three elements: Global Budgets; Health Based Allocation Method (HBAM); and Quality-Based Procedures (QBP). This evaluation focuses on implementation of QBPs, a procedure/diagnosis-specific funding approach involving a pre-set price per episode of care coupled with best practice clinical pathways. We examined whether or not there was consensus in understanding of the program theory underpinning QBPs and how this may have influenced full and effective implementation of this innovative funding model. METHODS: We undertook a formative evaluation of QBP implementation. We used an embedded case study method and in-depth, one-on-one, semi-structured, telephone interviews with key informants at three levels of the health care system: Designers (those who designed the QBP policy); Adoption Supporters (organizations and individuals supporting adoption of QBPs); and Hospital Implementers (those responsible for QBP implementation in hospitals). Thematic analysis involved an inductive approach, incorporating Framework analysis to generate descriptive and explanatory themes that emerged from the data. RESULTS: Five main findings emerged from our research: (1) Unbeknownst to most key informants, there was neither consistency nor clarity over time among QBP designers in their understanding of the original goal(s) for hospital funding reform; (2) Prior to implementation, the intended hospital funding mechanism transitioned from ABF to QBPs, but most key informants were either unaware of the transition or believe it was intentional; (3) Perception of the primary goal(s) of the policy reform continues to vary within and across all levels of key informants; (4) Four years into implementation, the QBP funding mechanism remains misunderstood; and (5) Ongoing differences in understanding of QBP goals and funding mechanism have created challenges with implementation and difficulties in measuring success. CONCLUSIONS: Policy drift and policy layering affected both the goal and the mechanism of action of hospital funding reform. Lack of early specification in both policy goals and hospital funding mechanism exposed the reform to reactive changes that did not reflect initial intentions. Several challenges further exacerbated implementation of complex hospital funding reforms, including a prolonged implementation schedule, turnover of key staff, and inconsistent messaging over time. These factors altered the trajectory of the hospital funding reforms and created confusion amongst those responsible for implementation. Enacting changes to hospital funding policy through a process that is transparent, collaborative, and intentional may increase the likelihood of achieving intended effects.

Defining Health Profession Regulators' Roles in the Canadian Healthcare System.

Health professions regulation today faces a myriad of challenges, due to both the perceived performance of regulatory colleges, how health systems have evolved, and even larger political and economic shifts such as the renegotiation of NAFTA. In this issue of Healthcare Papers, Wilkie and Tzountzouris (2017) describe the work of the College of Medical Laboratory Technologists of Ontario (CMLTO) to redefine professionalism in the context of these challenges. Their paper, and the comments of the responding authors in this issue highlight that there, is an overarching perception that health regulatory structures - across a range of professions - are not working as effectively as they should. Across this issue of Healthcare Papers, attention is drawn to the fact that more can be done to improve both the function and perception of professional regulatory bodies. However, each paper presents a different approach to how improvements in function and perception are possible.

Getting to Now: The Challenge of Stimulating Innovation in Complex Systems.

In this issue of Healthcare Papers, Anne Snowdon clearly articulates the elements necessary for a health system innovation agenda. Although these steps are not easy - witness the slow progress of the adoption and diffusion of innovation across Canadian healthcare - they are relatively simple and provide a nice counterpoint to our usual concerns about how to effect change within the complex environment of healthcare. The commentators in this issue re-enforce the importance of particular elements or the feasibility of taking some of the first hard steps.

Understanding relevance of health research: considerations in the context of research impact assessment.

BACKGROUND: With massive investment in health-related research, above and beyond investments in the management and delivery of healthcare and public health services, there has been increasing focus on the impact of health research to explore and explain the consequences of these investments and inform strategic planning. Relevance is reflected by increased attention to the usability and impact of health research, with research funders increasingly engaging in relevance assessment as an input to decision processes. Yet, it is unclear whether relevance is a synonym for or predictor of impact, a necessary condition or stage in achieving it, or a distinct aim of the research enterprise. The main aim of this paper is to improve our understanding of research relevance, with specific objectives to (1) unpack research relevance from both theoretical and practical perspectives, and (2) outline key considerations for its assessment. APPROACH: Our approach involved the scholarly strategy of review and reflection. We prepared a draft paper based on an exploratory review of literature from various fields, and gained from detailed and insightful analysis and critique at a roundtable discussion with a group of key health research stakeholders. We also solicited review and feedback from a small sample of expert reviewers. CONCLUSIONS: Research relevance seems increasingly important in justifying research investments and guiding strategic research planning. However, consideration of relevance has been largely tacit in the health research community, often depending on unexplained interpretations of value, fit and potential for impact. While research relevance seems a necessary condition for impact - a process or component of efforts to make rigorous research usable - ultimately, relevance stands apart from research impact. Careful and explicit consideration of research relevance is vital to gauge the overall value and impact of a wide range of individual and collective research efforts and investments. To improve understanding, this paper outlines four key considerations, including how research relevance assessments (1) orientate to, capture and compare research versus non-research sources, (2) consider both instrumental versus non-instrumental uses of research, (3) accommodate dynamic temporal-shifting perspectives on research, and (4) align with an intersubjective understanding of relevance.